Goldfish are actually fantastically dirty. When I kept orandas, lionheads, etc., I ended up doing it in about a 75-gal tank with HUGE canister-style filters like you were doing a reef tank or something. This was for about 10-12 fish (albeit the large, tennis-ball size ones). Even then, managing the water quality perfectly took a lot of work.

Cardinal tetras or something like that in a very small tank would be nice, but they don't have the personality of a Betta. I've got a Betta in a 5-gal tank right on my kitchen counter (which is a giant amount of space for that fish). You could do it just as well in a 2-gal tank or acrylic cube. I only need a very small heater and a very small submersible filter. It's practically zero work, mostly wiping the algae off the glass.

You want some floating aquatic plants with your Betta -- they love to hide and lurk like they're a vicious moray eel or something. For the same reason, some kind of hollow log fishtank sculpture on the bottom of the tank would be advisable.

Bettas can have about a 4-year life span in a good environment. This is my third kitchen Betta.